Traceability Gap Analysis for Assessing the Conformance of Software Traceability to Relevant Guidelines

Many guidelines for safety-critical industries such as aeronautics, medical devices, and railway communications, specify that traceability must be used to demonstrate that a rigorous process has been followed and to provide evidence that the system is safe for use. In practice, there is a gap between what is prescribed by guidelines and what is implemented in practice, making it difficult for organizations and certifiers to fully evaluate the safety of the software system. We present an approach, which parses a guideline to extract a Traceability Model depicting software artifact types and their prescribed traces. It then analyzes the traceability data within a project to identify areas of traceability failure. Missing traceability paths, redundant and/or inconsistent data, and other problems are highlighted. We used our approach to evaluate the traceability of seven safety-critical software systems and found that none of the evaluated projects contained traceability that fully conformed to its relevant guidelines. 1 Motivation and Traceability Challenges Developing safety-critical systems is a challenging process. Required features must be delivered in a way that ensures that the system is safe for use. Therefore, stringent guidelines must be met before the systems can be certified for use. Such guidelines typically prescribe activities, deliverable documents, and quality criteria focused around the software development lifecycle. As one important quality criterion, most guidelines mandate traceability. For example, the aviation guideline DO-178B/C mandates traceability from requirements to design, source code and executable object code. In practice, traceability is achieved through trace links, defined as “specified associations between a pair of artifacts, one comprising the source artifact and one comprising the target artifact” [CHGHH14]. It is important for demonstrating that a software mitigates all identified safety risks and a rigorous software development process has been followed. However, organizations struggle to establish complete traceability [RMK13]. A prior analysis of submissions to the US Food and Drug Administration (FDA) as part of the medical device approval process, showed a significant traceability gap between the traceability expectations as laid out in the FDA’s “Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices”, and the traceability data documented in the submissions [MJZCH13].